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COMBINING MULTIMEDIA SERVICES WITH TRADITIONAL TELEPHONY 

Field of the Invention 

[0001] The present Invention relates to facilitating multimedia services, and 
in particular, associating multimedia services with traditional telephony 
services in an efficient manner. 

Background of the Invention 

[0002] Traditional telephony services provided by digital switches, such as 
digital multiplexing switches, have reached their functional limits with existing 
user interfaces, which essentially are telephone sets having limited displays 
and simple keypads. Further, the telephone sets have limited bandwidth. 
Over newer packet networks, multimedia services are flourishing and are 
capable of exploiting the capabilities of advanced user terminals, desktop 
computers, and network appliances. 

[0003] Currently, the vast majority of voice telephony is provided, at least 
in part, by traditional circuit-switched networks. Given the extensive 
infrastructure, reliability, and quality of service, the traditional telephony 
systems are likely to remain a significant part of communications for the 
foreseeable future. Unfortunately, there has been difficulty integrating voice 
sessions over the traditional telephony network with multimedia sessions over 
packet networks. Users prefer the traditional telephony network for voice, yet 
the voice network is unacceptable for facilitating advanced multimedia 
services, such as screen sharing, video conferencing, and the like. Given the 
unique strengths of the respective communication systems, there is a need for 
an efficient and economical way to facilitate interworking between the 
networks. There is a further need to facilitate such interworking without 
requiring significant changes to the traditional telephony or packet-switched 
infrastructures and communication protocols. 

Summarv of the Invention 

[0004] The present Invention provides a combined user agent (CUA) to 
represent a telephone supported by a circuit-switched telephony switch and a 
packet-based media device as an integrated group to other network entities. 
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The CUA is configured to facilitate the necessary call signaling to establish 
and control a voice call via the telephony switch, as well as the session 
control signaling necessary to establish and control a media session with the 
media device. Accordingly, the telephone and media device appear to the 
network devices as a single device having voice and media capabilities, 
wherein the voice capabilities are controlled In part by the telephony switch. 
[0005] Although the telephony switch uses circuit-switched 
communications, the telephone may be a circuit-switched telephone 
supported by the switch or may be a packet-based telephone, which is 
supported by a gateway supported the telephony switch. Further, the call 
signaling may take any form acceptable by the telephony switch to facilitate 
call processing and control. For example, the call signaling may conform to 
an intelligent network protocol and take place in part over an intelligent 
network primarily dedicated for call signaling. 

[0006] In one embodiment, the session initiation protocol (SIP) is used to 
facilitate communications between the CUA and other SIP devices as well as 
the media devices represented by the CUA. Preferably, the voice call and 
media session are associated with one another using the CUA, and 
information about or related to the voice call and media session may be 
shared with applications participating in the voice call or media session. The 
applications may include video conferencing, audio streaming, video 
streaming, information streaming, voicemail, email, gaming, advertising, 
screen sharing, instant messaging, and the like. 

[0007] Those skilled In the art will appreciate the scope of the present 
invention and realize additional aspects thereof after reading the following 
detailed description of the preferred embodiments in association with the 
accompanying drawing figures. 

Brief Description of the Drawing Figures 

[0008] The accompanying drawing figures incorporated in and forming a 
part of this specification illustrate several aspects of the invention, and 
together with the description serve to explain the principles of the invention. 
[0009] FIGURE 1 is a communication environment according to a first 
embodiment of the present invention. 
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[0010] FIGURE 2 is a communication environment according to a second 
embodiment of the present invention. 

[0011] FIGURE 3 is a communication environment according to a third 
embodiment of the present invention. 

[0012] FIGURE 4 Is a block representation of a combined user agent 
according to one embodiment of the present invention. 
[0013] FIGURES 5A, 5B, and 5C are a communication flow diagram 
outlining an exemplary technique for associating a voice sharing and a screen 
sharing multimedia session in a communication environment as illustrated in 
Figure 3. 

Detailed Descriotion of the Preferred Embodiments 
[0014] The embodiments set forth below represent the necessary 
information to enable those skilled in the art to practice the Invention and 
illustrate the best mode of practicing the invention. Upon reading the 
following description in light of the accompanying drawing figures, those 
skilled in the art will understand the concepts of the invention and will 
recognize applications of these concepts not particularly addressed herein. It 
should be understood that these concepts and applications fall within the 
scope of the disclosure and the accompanying claims. 
[0015] The present invention facilitates efficient unification of parallel voice 
and multimedia sessions, wherein the voice session takes place in part over a 
traditional circuit-switched telephony network. A call signaling agent is 
created to control packet-based multimedia sessions, as well as to control call 
signaling at a traditional telephony switch, for a telephone, and an associated 
multimedia device, such as a computer. The multi-functional call processing 
agent, typically refen-ed to as a combined user agent (CUA), can effectively 
establish multimedia sessions with the multimedia device and voice calls with 
the telephone. 

[0016] With reference to Figure 1 , a communication environment 10 
according to one embodiment is illustrated. The communication environment 
10 may include a packet network 12 including a CUA 14 and a supporting 
proxy 16. The CUA 14 acts as a virtual agent for a computing device, such as 
a computer 18, which is capable of supporting multimedia sessions. The 
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computer 18 may connect to the packet network 12 via a network access 20, 
which may include a local area network (LAN), frame relay, digital subscriber 
line (DSL), cable, or the like. 

[0017] The CUA 14 also acts as an agent for a traditional telephony 
device, such as a telephone 22, which is supported by a telephony switch 24 
such as a digital multiple switch, public branch exchange (PBX), or the like, 
that is capable of providing circuit-switched communications between the 
telephone 22 and other telephony devices. To allow the CUA 14 to interact 
with and control the telephony switch 24, legacy access 26 is provided 
between the packet network 12 and the telephony switch 24. The legacy 
access 26 may be provided by existing intelligent networks (IN), including the 
advanced intelligent network (AIN), SIP-T capable networks, TAPI networks, 
and the like, that provide access to telephony switches 24 to facilitate call 
signaling. The CUA 14 is configured to establish multimedia sessions over 
network access 20 with the computer 18 as well as provide call signaling for 
the telephone 22 through the telephony switch 24 via the legacy access 26. 
[001 8] Although the concepts of the present invention are applicable to 
various communication environments and related protocols, the present 
invention is preferably implemented using the session initiation protocol, 
which is commonly referred to as SIP. The specification for SIP is provided in 
the Internet Engineering Task Force's Request for Comments (RFC) 2543: 
Session Initiation Protocol Internet Draft, which is incorporated herein by 
reference in its entirety. In general, SIP is used to establish media sessions 
between any number of endpoints. Typically, these endpoints may support 
any number or combination of data, audio, and voice media sessions, 
depending on the configuration of the device. A SIP endpoint is capable of 
running an application, typically referred to as a user agent (UA), which is 
capable of facilitating media sessions using SIP. In certain embodiment, user 
agents may register their ability to establish sessions with a SIP proxy by 
sending "REGISTER" messages to the SIP proxy. The REGISTER message 
informs the SIP proxy of the SIP universal resource locator (URL) that 
identifies the user agent to the SIP network. The REGISTER message also 
contains information about how to reach specific user agents over the SIP 
network, typically by providing the Internet Protocol (IP) address and port that 
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the user agent will use for SIP sessions. When a user agent wants to 
establish a session with another user agent, the user agent initiating the 
session may send an INVITE message to the SIP proxy and specify the target 
user agent in the TO header of the INVITE message. Identification of the user 
agent takes the form of a SIP URL. The SIP proxy will use the SIP URL in the 
TO header of the message to determine if the targeted user agent is 
registered with the SIP proxy. Generally the user name is unique within the 
name space of the specified domain. 

[0019] If the targeted user agent has registered with the SIP proxy, the SIP 
proxy will fonward the INVITE message directly to the targeted user agent. 
The targeted user agent will respond with a 200 OK message, and a session 
between the respective user agents will be established as per the message 
exchange required in the SIP specification. Media capabilities are passed 
between the two user agents of the respective endpoints as parameters 
embedded within the session setup messages, such as the INVITE, 200 OK, 
and acknowledgement (ACK) messages. Media capabilities may be 
exchanged in other messages, such as the SIP "INFO" message. Media 
capabilities are typically described using the session description protocol 
(SDP). Once respective endpoints are in an active session with each other 
and have determined each other's capabilities, the specified media content 
may be exchanged during an appropriate media session. 
[0020] According to the Intemet Engineering Task Force's RFC 2543, a 
user agent is an application that contains both a user agent client and a user 
agent server. A user agent client generally refers to a client application that 
initiates SIP requests, wherein a user agent sen/er is an application that 
contacts the user when a SIP request is received, and returns a response on 
behalf of the user. Typically, the response accepts, rejects, or redirects the 
received request. 

[0021] The present invention provides for the CUA 14, which is an 
application, program, or function that acts on behalf of a multimedia client, 
provided by the computer 18 or like device, and a telephone 22. The 
computer 18 may have a SIP user agent, which is represented by the CUA 
14. In essence, the CUA 14 will function to control call signaling to effect 
voice sessions between the telephone 22 and other devices via the telephony 
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switch 24, and other multimedia sessions via the computer 18. Notably, the 
CUA 14 can effectively facilitate the integration and association of voice and 
other multimedia sessions provided by both devices. To devices needing to 
establish sessions with either the computer 18 or the telephone 22 in a SIP 
environment, the CUA 14 presents both devices as a single device having 
voice and other multimedia capabilities. The other devices need not know 
that the telephone 22 resides on a circuit-switched network. 
[0022] In one embodiment, the CUA 14 will cooperate with the proxy 1 6, 
which is configured as a SIP proxy, in traditional fashion to establish 
multimedia sessions involving the computer 18. The sessions will be 
supported across the network access 20 in the packet network 12. With 
respect to voice communications via the telephone 22, the voice path may 
take various routes, remaining entirely within the public switched telephone 
network (PSTN), or venturing into the packet network 12 to communicate with 
a packet-switched telephony device or simply to bridge to another circuit- 
switched network. 

[0023] The communication environment 10 illustrated in Figure 1 illustrates 
voice communications between the telephone 22 and a voice-capable packet- 
switched device 30, such as an IP telephone 30A or a computer 30B. The 
packet-switched devices 30 are connected to the packet network 12 via a 
local area network 32 to support packet-switched communications. Since the 
telephone 22 and telephony switch 24 facilitate circuit-switched 
communications, a gateway (GW) 28 is integrated with the telephony switch 
24 or provided as a separate device (as shown) to convert circuit-switched 
communications to packet-switched communications capable of being 
transported over the packet network 1 2 to the desired packet-switched device 
30. The connection between the telephony switch 24 and the gateway 28 
may be any traditional telephony line or trunk. The gateway 28 appears to the 
telephony switch 24 as another switching device supporting a variety of 
telephone numbers, which are associated with the packet-switched devices 
30. Typically, the time-division multiplexed circuit-switched communications 
are converted into packets to facilitate voice communications over the Internet 
Protocol (VoIP). Importantly, although the voice session spans the circuit- 
switched and packet-switched networks, the CUA 14 represents the 
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telephone 22 to the devices with which it communicates. In essence, the 
telephone 22 and the computer 18 form a combined legacy client 34, which is 
represented by the CUA 14. 

[0024] With reference to Figure 2, another CUA 14A is provided to support 
a computer 18A via network access 20A and a circuit-switched telephone 
22A, which is also supported by telephony switch 24A. The CUA 14A 
provides call signaling to the telephony switch 24A via legacy access 26A in a 
similar manner as that described above. Further, the telephony switch 24A is 
associated with a gateway 28A to convert circuit-switched communications 
into packet-switched communications for transport over the packet network 
12. The communication environment 10 in Figure 2 illustrates the circuit- 
switched communications between telephone 22 and telephone 22A during a 
voice session being routed over the packet network 12 between the gateways 
28 and 28A. Multimedia sessions other than voice between computers 18 
and 18A may be established in traditional fashion over the packet network 12. 
Again, the CUA 14 represents the combined legacy client 34 containing the 
computer 18 and telephone 22, whereas the CUA 14A supports the combined 
legacy client 34A, which is made up of the computer 18A and telephone 22A. 
In this embodiment, the CUA 14A is supported by the proxy 16. 
[0025] With reference to Figure 3, the CUA 14A may be supported by a 
separate proxy 16A. Further, the communications between the telephone 22 
and telephone 22A during a voice session may be supported entirely within 
the PSTN 36. The CUAs 14 and 14A may provide call signaling not only to 
the respective telephony switches 24 and 24A, but to other network elements 
within the PSTN 36. In short, the CUAs 14 and 14A are configured to provide 
the necessary call signaling to establish voice sessions that are supported at 
least partially over the circuit-switched network of the PSTN 36, as well as 
multimedia sessions with the computer 18 over the packet network 12. 
[0026] As illustrated in Figure 4, the combined user agent 14 is preferably 
implemented in a control system 36 associated with a packet network 
Interface 38 for communicating over the packet network 12. The control 
system 36 will support software applications providing a legacy adapter 40, a 
multimedia client adapter 42, and the basic CUA logic 44. The legacy adapter 
40 will provide the necessary protocol adaptation and call signaling control 
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necessary to control the telephony switch 24 in light of SIP or related 
protocols for establishing media sessions. The multimedia client adapter 42 is 
used to support sessions with the associated computer 18 or like multimedia 
device. The multimedia client adapter 42 may provide protocol adaptation as 
necessary to establish the media sessions or a SIP implementation, wherein 
when the computer 18 emulates a SIP client, the CUA logic 44 will simply 
represent the multimedia capabilities of the computer 18 for the non-voice 
media sessions. The CUA logic 44 will also cooperate with the legacy 
adapter 40 to provide the necessary call signaling for the telephony switch 24 
to control voice communications with the telephone 22. Accordingly, the CUA 
logic 44 cooperates with the legacy adapter 40 and the multimedia client 
adapter 42 to provide an interface to the computer 18 as well as an interface 
to the telephony switch 24, and an interface for communications with other 
devices, such as the proxy 16. 

[0027] Turning now to Figures 5A-5C, an exemplary communication flow is 
shown for establishing a voice session and a screen sharing session between 
the telephones 22 and 22A and the computers 18 and 18A, respectively, in 
the communication environment 10 of Figure 3. In a SIP environment, user 1 
and user 2 will log in to their respective computers 18 and 18A, which will 
register with their respective proxies 16 and 16A. Accordingly, user 1 will log 
in to computer 18 (step 100), which will send a REGISTER message to CUA 
14 with the URL for the computer 1 8 and the telephone number for the 
associated telephone 22 (step 102). The CUA 14 will provide this information 
to the supporting proxy 16 in a REGISTER message (step 104). Likewise, 
user 2 will log in to computer 18A (step 106), which will provide the URL for 
computer 18A and the telephone number for the associated telephone 22A to 
the CUA 14A (step 108), which will send another REGISTER request to the 
associated proxy 16A (step 110). 

[0028] The CUA 14 will then arm the intelligent network triggers by sending 
an appropriate message to the telephony switch 24 (step 1 12). The CUA 14A 
will also arm the intelligent network triggers of telephony switch 24A (step 
114). At this point, the combined legacy clients 34 and 34A are properly 
registered and ready for initiating voice and other multimedia sessions. 
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[0029] Assume that the session is Initiated by user 1 dialing the telephone 
number of telephone 22A (step 116). Accordingly, the telephone 22 will alert 
the telephony switch 24 of an off-hook status and report the digits dialed (step 
118). In this embodiment, messages between the telephones 22 and 22A 
and the telephony switches 24 and 24A are transported in traditional PSTN 
fashion. The telephony switch 24 will start processing the call as usual and 
will reach a point where it will encounter the IN trigger set in step 112. The 
telephony switch 24 will respond by sending an information-analyzed 
message to the CUA 14 indicating that user 1 is dialing the number of 
telephone 22A (step 120). The CUA 14 will log the call (step 122) and send 
an INFO message to the computer 18 indicating that the call is proceeding 
(step 124). The computer 18 may use this information to retrieve relevant 
Information like appointments, emails, documents, etc. which may be 
associated to the user associated with the telephone number of telephone 
22A. The CUA 14 will next send a CONTINUE message to the telephony 
switch 24 providing the URL and phone number for user 1 (step 126). Upon 
receiving the CONTINUE message, the telephony switch 24 will continue call 
processing (step 128) until it reaches the point where it determines that the 
call needs to proceed via switch 24 A. To do so, switch 24 will send a SIP 
trunking (SIP-T') INVITE message to telephony switch 24A (step 130). The 
INVITE message will include the URL and telephone number for user 1 as 
well as the dialed number corresponding to telephone 22A (user 2). in 
return, the telephony switch 24A will provide call processing (step 132) and 
eventually encounter the intelligent network trigger that was armed in step 
114. This will cause the telephony switch 24A to send a message indicating 
an attempt to terminate a call for telephone 22A to CUA 14A (step 134). The 
message received by the CUA 14A will include the URL and telephone 
number for user 1 and the telephone number for telephone 22A. 
[0030] The CUA 14A will perform a lookup using the telephone number for 
telephone 22A, add the corresponding URL for the computer 18A and forward 
an INVITE message including the URLs and telephone numbers for both user 
1 and user 2 to the proxy 16A (step 136). The proxy 16A will provide any 
necessary services (step 138), such as call screening, and, assuming that the 
call can proceed to telephone 22A, will respond with an INVITE message 
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back to CUA 14A (step 140), which will trigger the CUA 14A to send an INFO 
message indicating that user 1 is calling user 2 on telephone 22A to the 
computer 18Afor reference (step 142). Further, the CUA 14A will send a 
CONTINUE message identifying the URL and telephone number for user 2 to 
the telephony switch 24A (step 144). The telephony switch 24A will provide 
further call processing (step 146) and apply ringing to the telephone 22A of 
user 2 (step 148). The telephony switch 24A will also send a SIP-T 180 
RINGING message to telephony switch 24 to alert the telephony switch 24 
that telephone 22A is being rung (step 150). The telephony switch 24 will 
enable the voice path toward telephone 22 of user 1 , which will result in a 
one-way voice path and the audible ringing sent by switch 24A to be heard at 
telephone 22 (step 152). 

[0031] At some point, user 2 will answer their call (step 1 54), which will 
prompt the telephone 22A to Indicate an OFFHOOK condition, which is 
detected by the telephony switch 24A (step 156). In response, the telephony 
switch 24A will perform additional call processing and will encounter another 
intelligent network trigger armed in step 1 14. This trigger will cause telephony 
switch 24A to send an ANSWER message to the CUA 14A (step 158), which 
will send a message to computer 18A that a call is in progress (step 160). 
Further, the telephony switch 24A will send a SIP-T 200 OK message 
including the URLs and telephone numbers for both users 1 and 2 to 
telephony switch 24 (step 162), which will enable a two-way voice path. The 
telephony switch 24 will also perform additional call processing and will 
encounter another intelligent network trigger armed in step 112. The 
telephony switch 24 will send an ANSWER message including the URL and 
telephone number of user 2 to the CUA 14 (step 164), which will send a 
message to computer 18 that a call is in progress (step 166). At this point, a 
call is established between telephone 22 and telephone 22A (step 168). 
[0032] Once the voice session is established between the telephones 22 
and 22A, a screen sharing application is initiated by user 1 at computer 18. 
Accordingly, computer 18 will send a message to the CUA 14 indicating that it 
wants to initiate screen sharing (step 170). The CUA 14 will send a SIP 
INVITE message to the supporting proxy 16 indicating that a screen sharing 
session is desired between computers 18 and 18A having the URLs 
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associated with both users (step 172). The proxy 16 will forward the INVITE 
message to the SIP proxy 16A (step 174), which will further forward the 
INVITE message to the CUA 14A (step 176). The CUA 14A will send a 
message offering screen sharing to computer 18A (step 178). Computer 18A, 
assuming that screen sharing is accepted, will send an acknowledgement 
(ACK) of the screen sharing to the CUA 14A (step 180), which will forward a 
similar ACK message to CUA 14 (step 182), which will further forward the 
ACK message to computer 18 (step 184). Upon completion of the ACK 
messages, a screen sharing session is established between computer 1 8 and 
computer 18A (step 186). 

[0033] As seen from the above, the CUA 14 operates on behalf of the 
supported telephone 22 and computer 18 to facilitate media sessions as a 
packet-switched entity. For voice sessions, the CUA 14 effectively controls 
call signaling for the supporting telephony switch 24 to facilitate voice 
sessions without knowledge by other network devices. 
[0034] Although a screen sharing session is illustrated, those skilled in the 
art will recognize that concepts of the present invention are equally applicable 
to video conferencing, audio streaming, video streaming, information 
streaming, voicemail, email, gaming, advertising, instant messaging, or any 
other desired multimedia session that may benefit by having an affiliated voice 
session. Further, although circuit-switched telephones, such as telephone 
22A, are disclosed and described, the telephony switch 24 may support 
devices that further support packet-switched telephones. The invention is 
equally applicable and beneficial to such configurations, where a voice 
session or like media session must be carried out at least in part through a 
telephony switching device. In operation, the CUA 14 is preferably configured 
to interact with the computer 18 to enhance the functionality and usefulness of 
applications supporting the voice and multimedia sessions. For example, the 
CUA 14 can provide information to applications on the computer 18 indicating 
that a voice session is in progress, and also provide information about the 
voice session, such as with whom the voice session is occurring and any 
associated information, such as a telephone number of a participating device 
or user. These features are particularly useful for conference calls, where 
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presentations must be shared and it is beneficial to identify the participating 
parties. 

[0035] Those skilled in the art will recognize improvements and 
modifications to the preferred embodiments of the present invention. For 
example, the switch 24 may be a wireless switch supporting wireless 
communication via a base station with a mobile terminal, such as a mobile 
telephone, personal digital assistant (PDA), or the like. Accordingly, the voice 
connection for the voice session may be established over a wireless link. All 
such improvements and modifications are considered within the scope of the 
concepts disclosed herein and the claims that follow. 



